Why this project

• Get back into the books and notes to refresh on concepts and software

• Refresh on and practice using R/RStudio

• Experiment with flexdashboards to see if I might want/how to incorporate this as part a workflow process

• Work with a relevant data set

• Learning new things about flexdashboards in R/Rmarkdown (Ex: using html for picture sizing & placement)

Important Note(s)

• Source code for this project can be found on my GitHub repo at HR_Analytics.

• This is best viewed on a wide-screen monitor. A 24" or larger monitor is recommended.

• After opening this file, expand or maximize the window to properly view it.

• A small, or reduced, window size causes the top tabs to move to a second line in the header row. This collapses the page contents in a manner that hides various window/section headers, etc.

Sample image

Image Source: https://images.techhive.com/images/article/2016/09/data_science_classes-100682563-large.jpg

Experimental section

This section demonstrates showing a code block without the result(s).

a <- 2 + 2

b <- function(x){
  print(x^2)
}

b(a)

rm(a, b) # clean memory

Data Source

IBM HR Analytics Employee Attrition & Performance
Downloaded from: https://www.kaggle.com/pavansubhasht/ibm-hr-analytics-attrition-dataset

• Data is fictional - created by IBM data scientists

• Insight considerations (general):

  • Predict attrition
  • What factors contribute to attrition
  • Once identify factors contributing to attrition, deep-dive and/or comparisons to develop understanding of those factors

Data Sample (scrollable)

Data Overview

Data Dictionary (scrollable)

\(attrition\)
0 ‘No’ 1 ‘Yes’

\(education\)
1 ‘Below College’ 2 ‘College’ 3 ‘Bachelor’ 4 ‘Master’ 5 ‘Doctor’

\(environmentsatisfaction\)
1 ‘Low’ 2 ‘Medium’ 3 ‘High’ 4 ‘Very High’

\(jobinvolvement\)
1 ‘Low’ 2 ‘Medium’ 3 ‘High’ 4 ‘Very High’

\(joblevel\)
Ordinal levels represented by 1, 2, 3, 4, 5. No further meaning known.

\(jobsatisfaction\)
1 ‘Low’ 2 ‘Medium’ 3 ‘High’ 4 ‘Very High’

\(performancerating\)
1 ‘Low’ 2 ‘Good’ 3 ‘Excellent’ 4 ‘Outstanding’

\(relationshipsatisfaction\)
1 ‘Low’ 2 ‘Medium’ 3 ‘High’ 4 ‘Very High’

\(stockoptionlevel\)
Ordinal levels represented by 0, 1, 2, 3. No further meaning known.

\(worklifebalance\)
1 ‘Bad’ 2 ‘Good’ 3 ‘Better’ 4 ‘Best’

Check X vs. Y Independence

• Here, we’ll check to see if there’s a relationship between (\(H_o: No\ relationship/independent)\)) each categorical variable and the response variable using contingency tables, \(\chi^2\), and \(p-values\). Where contingency tables have an ordinal variable w/ attrition (nominal - 2 levels) we will use the Cochran-Mantel-Haenszel (CMH) test, a linear trend test, since it will have more power. The results of the CMH test will be directly beneath the corresponding CrossTable and indicated by d.f. = 1.

• Use \(\chi^2\) test for independence for the following predictors w/ attrition:
  • \(businesstravel\)
  • \(department\)
  • \(educationfield\)
  • \(gender\)
  • \(jobrole\)
  • \(maritalstatus\)
  • \(overtime\)

• CMH test for the remaining categorical predictors vs. attrition

• The following predictors are independent of the response (i.e. \(p-value > 0.05\)). We will remove these predictors from the model since the response does not depend on these predictors.
  • \(gender\)
  • \(relationshipsatisfaction\)
  • \(worklifebalance\)

\(\chi^2\) tests (scrollable)

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
                  | attrition 
   businesstravel |         0 |         1 | Row Total | 
------------------|-----------|-----------|-----------|
       non-travel |        99 |         7 |       106 | 
                  |    88.591 |    17.409 |           | 
------------------|-----------|-----------|-----------|
    travel_rarely |       620 |       113 |       733 | 
                  |   612.614 |   120.386 |           | 
------------------|-----------|-----------|-----------|
travel_frequently |       141 |        49 |       190 | 
                  |   158.795 |    31.205 |           | 
------------------|-----------|-----------|-----------|
     Column Total |       860 |       169 |      1029 | 
------------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  20.13083     d.f. =  2     p =  4.252523e-05 


 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
                       | attrition 
            department |         0 |         1 | Row Total | 
-----------------------|-----------|-----------|-----------|
       human resources |        33 |         8 |        41 | 
                       |    34.266 |     6.734 |           | 
-----------------------|-----------|-----------|-----------|
research & development |       583 |        91 |       674 | 
                       |   563.304 |   110.696 |           | 
-----------------------|-----------|-----------|-----------|
                 sales |       244 |        70 |       314 | 
                       |   262.430 |    51.570 |           | 
-----------------------|-----------|-----------|-----------|
          Column Total |       860 |       169 |      1029 | 
-----------------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  12.35835     d.f. =  2     p =  0.002072139 


 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
                 | attrition 
  educationfield |         0 |         1 | Row Total | 
-----------------|-----------|-----------|-----------|
 human resources |        15 |         4 |        19 | 
                 |    15.879 |     3.121 |           | 
-----------------|-----------|-----------|-----------|
   life sciences |       360 |        63 |       423 | 
                 |   353.528 |    69.472 |           | 
-----------------|-----------|-----------|-----------|
       marketing |        94 |        27 |       121 | 
                 |   101.127 |    19.873 |           | 
-----------------|-----------|-----------|-----------|
         medical |       275 |        42 |       317 | 
                 |   264.937 |    52.063 |           | 
-----------------|-----------|-----------|-----------|
           other |        49 |         5 |        54 | 
                 |    45.131 |     8.869 |           | 
-----------------|-----------|-----------|-----------|
technical degree |        67 |        28 |        95 | 
                 |    79.397 |    15.603 |           | 
-----------------|-----------|-----------|-----------|
    Column Total |       860 |       169 |      1029 | 
-----------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  20.20982     d.f. =  5     p =  0.001141335 


 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
             | attrition 
      gender |         0 |         1 | Row Total | 
-------------|-----------|-----------|-----------|
      female |       359 |        63 |       422 | 
             |   352.692 |    69.308 |           | 
-------------|-----------|-----------|-----------|
        male |       501 |       106 |       607 | 
             |   507.308 |    99.692 |           | 
-------------|-----------|-----------|-----------|
Column Total |       860 |       169 |      1029 | 
-------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  1.164534     d.f. =  1     p =  0.2805271 

Pearson's Chi-squared test with Yates' continuity correction 
------------------------------------------------------------
Chi^2 =  0.9872404     d.f. =  1     p =  0.3204178 

 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
                          | attrition 
                  jobrole |         0 |         1 | Row Total | 
--------------------------|-----------|-----------|-----------|
healthcare representative |        83 |         5 |        88 | 
                          |    73.547 |    14.453 |           | 
--------------------------|-----------|-----------|-----------|
          human resources |        22 |         8 |        30 | 
                          |    25.073 |     4.927 |           | 
--------------------------|-----------|-----------|-----------|
    laboratory technician |       137 |        37 |       174 | 
                          |   145.423 |    28.577 |           | 
--------------------------|-----------|-----------|-----------|
                  manager |        74 |         4 |        78 | 
                          |    65.190 |    12.810 |           | 
--------------------------|-----------|-----------|-----------|
   manufacturing director |        84 |         7 |        91 | 
                          |    76.054 |    14.946 |           | 
--------------------------|-----------|-----------|-----------|
        research director |        60 |         1 |        61 | 
                          |    50.982 |    10.018 |           | 
--------------------------|-----------|-----------|-----------|
       research scientist |       182 |        39 |       221 | 
                          |   184.704 |    36.296 |           | 
--------------------------|-----------|-----------|-----------|
          sales executive |       183 |        43 |       226 | 
                          |   188.882 |    37.118 |           | 
--------------------------|-----------|-----------|-----------|
     sales representative |        35 |        25 |        60 | 
                          |    50.146 |     9.854 |           | 
--------------------------|-----------|-----------|-----------|
             Column Total |       860 |       169 |      1029 | 
--------------------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  63.88879     d.f. =  8     p =  8.000966e-11 


 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
              | attrition 
maritalstatus |         0 |         1 | Row Total | 
--------------|-----------|-----------|-----------|
       single |       245 |        84 |       329 | 
              |   274.966 |    54.034 |           | 
--------------|-----------|-----------|-----------|
      married |       405 |        63 |       468 | 
              |   391.137 |    76.863 |           | 
--------------|-----------|-----------|-----------|
     divorced |       210 |        22 |       232 | 
              |   193.897 |    38.103 |           | 
--------------|-----------|-----------|-----------|
 Column Total |       860 |       169 |      1029 | 
--------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  31.01857     d.f. =  2     p =  1.838245e-07 


 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
             | attrition 
    overtime |         0 |         1 | Row Total | 
-------------|-----------|-----------|-----------|
          no |       664 |        78 |       742 | 
             |   620.136 |   121.864 |           | 
-------------|-----------|-----------|-----------|
         yes |       196 |        91 |       287 | 
             |   239.864 |    47.136 |           | 
-------------|-----------|-----------|-----------|
Column Total |       860 |       169 |      1029 | 
-------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  67.73148     d.f. =  1     p =  1.873485e-16 

Pearson's Chi-squared test with Yates' continuity correction 
------------------------------------------------------------
Chi^2 =  66.19615     d.f. =  1     p =  4.082086e-16 

 

CMH tests (scrollable)

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
             | attrition 
   education |         0 |         1 | Row Total | 
-------------|-----------|-----------|-----------|
           1 |        96 |        25 |       121 | 
             |       101 |        20 |           | 
-------------|-----------|-----------|-----------|
           2 |       160 |        33 |       193 | 
             |       161 |        32 |           | 
-------------|-----------|-----------|-----------|
           3 |       341 |        73 |       414 | 
             |       346 |        68 |           | 
-------------|-----------|-----------|-----------|
           4 |       234 |        35 |       269 | 
             |       225 |        44 |           | 
-------------|-----------|-----------|-----------|
           5 |        29 |         3 |        32 | 
             |        27 |         5 |           | 
-------------|-----------|-----------|-----------|
Column Total |       860 |       169 |      1029 | 
-------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  5.528328     d.f. =  4     p =  0.2372506 


 

     Chisq         Df       Prob 
4.36088337 1.00000000 0.03677323 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
                        | attrition 
environmentsatisfaction |         0 |         1 | Row Total | 
------------------------|-----------|-----------|-----------|
                      1 |       144 |        55 |       199 | 
                        |       166 |        33 |           | 
------------------------|-----------|-----------|-----------|
                      2 |       190 |        30 |       220 | 
                        |       184 |        36 |           | 
------------------------|-----------|-----------|-----------|
                      3 |       262 |        48 |       310 | 
                        |       259 |        51 |           | 
------------------------|-----------|-----------|-----------|
                      4 |       264 |        36 |       300 | 
                        |       251 |        49 |           | 
------------------------|-----------|-----------|-----------|
           Column Total |       860 |       169 |      1029 | 
------------------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  23.95468     d.f. =  3     p =  2.553014e-05 


 

       Chisq           Df         Prob 
1.602041e+01 1.000000e+00 6.266329e-05 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
               | attrition 
jobinvolvement |         0 |         1 | Row Total | 
---------------|-----------|-----------|-----------|
             1 |        43 |        23 |        66 | 
               |        55 |        11 |           | 
---------------|-----------|-----------|-----------|
             2 |       212 |        48 |       260 | 
               |       217 |        43 |           | 
---------------|-----------|-----------|-----------|
             3 |       513 |        89 |       602 | 
               |       503 |        99 |           | 
---------------|-----------|-----------|-----------|
             4 |        92 |         9 |       101 | 
               |        84 |        17 |           | 
---------------|-----------|-----------|-----------|
  Column Total |       860 |       169 |      1029 | 
---------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  22.44157     d.f. =  3     p =  5.278864e-05 


 

       Chisq           Df         Prob 
1.856557e+01 1.000000e+00 1.641587e-05 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
             | attrition 
    joblevel |         0 |         1 | Row Total | 
-------------|-----------|-----------|-----------|
           1 |       284 |       102 |       386 | 
             |       323 |        63 |           | 
-------------|-----------|-----------|-----------|
           2 |       327 |        40 |       367 | 
             |       307 |        60 |           | 
-------------|-----------|-----------|-----------|
           3 |       126 |        20 |       146 | 
             |       122 |        24 |           | 
-------------|-----------|-----------|-----------|
           4 |        76 |         3 |        79 | 
             |        66 |        13 |           | 
-------------|-----------|-----------|-----------|
           5 |        47 |         4 |        51 | 
             |        43 |         8 |           | 
-------------|-----------|-----------|-----------|
Column Total |       860 |       169 |      1029 | 
-------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  48.98864     d.f. =  4     p =  5.870761e-10 


 

       Chisq           Df         Prob 
3.199786e+01 1.000000e+00 1.543427e-08 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
                | attrition 
jobsatisfaction |         0 |         1 | Row Total | 
----------------|-----------|-----------|-----------|
              1 |       156 |        41 |       197 | 
                |       165 |        32 |           | 
----------------|-----------|-----------|-----------|
              2 |       157 |        32 |       189 | 
                |       158 |        31 |           | 
----------------|-----------|-----------|-----------|
              3 |       269 |        55 |       324 | 
                |       271 |        53 |           | 
----------------|-----------|-----------|-----------|
              4 |       278 |        41 |       319 | 
                |       267 |        52 |           | 
----------------|-----------|-----------|-----------|
   Column Total |       860 |       169 |      1029 | 
----------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  5.834937     d.f. =  3     p =  0.1199229 


 

     Chisq         Df       Prob 
5.20628004 1.00000000 0.02250544 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
                         | attrition 
relationshipsatisfaction |         0 |         1 | Row Total | 
-------------------------|-----------|-----------|-----------|
                       1 |       158 |        45 |       203 | 
                         |       170 |        33 |           | 
-------------------------|-----------|-----------|-----------|
                       2 |       180 |        30 |       210 | 
                         |       176 |        34 |           | 
-------------------------|-----------|-----------|-----------|
                       3 |       277 |        47 |       324 | 
                         |       271 |        53 |           | 
-------------------------|-----------|-----------|-----------|
                       4 |       245 |        47 |       292 | 
                         |       244 |        48 |           | 
-------------------------|-----------|-----------|-----------|
            Column Total |       860 |       169 |      1029 | 
-------------------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  6.46917     d.f. =  3     p =  0.09088631 


 

    Chisq        Df      Prob 
2.3512264 1.0000000 0.1251845 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
                 | attrition 
stockoptionlevel |         0 |         1 | Row Total | 
-----------------|-----------|-----------|-----------|
               0 |       332 |       109 |       441 | 
                 |       369 |        72 |           | 
-----------------|-----------|-----------|-----------|
               1 |       375 |        41 |       416 | 
                 |       348 |        68 |           | 
-----------------|-----------|-----------|-----------|
               2 |       104 |         9 |       113 | 
                 |        94 |        19 |           | 
-----------------|-----------|-----------|-----------|
               3 |        49 |        10 |        59 | 
                 |        49 |        10 |           | 
-----------------|-----------|-----------|-----------|
    Column Total |       860 |       169 |      1029 | 
-----------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  41.07117     d.f. =  3     p =  6.315821e-09 


 

       Chisq           Df         Prob 
2.017610e+01 1.000000e+00 7.062985e-06 

 
   Cell Contents
|-------------------------|
|                       N |
|              Expected N |
|-------------------------|

 
Total Observations in Table:  1029 

 
                | attrition 
worklifebalance |         0 |         1 | Row Total | 
----------------|-----------|-----------|-----------|
              1 |        36 |        16 |        52 | 
                |        43 |         9 |           | 
----------------|-----------|-----------|-----------|
              2 |       199 |        41 |       240 | 
                |       201 |        39 |           | 
----------------|-----------|-----------|-----------|
              3 |       541 |        93 |       634 | 
                |       530 |       104 |           | 
----------------|-----------|-----------|-----------|
              4 |        84 |        19 |       103 | 
                |        86 |        17 |           | 
----------------|-----------|-----------|-----------|
   Column Total |       860 |       169 |      1029 | 
----------------|-----------|-----------|-----------|

 
Statistics for All Table Factors


Pearson's Chi-squared test 
------------------------------------------------------------
Chi^2 =  9.601838     d.f. =  3     p =  0.02227229 


 

     Chisq         Df       Prob 
3.05963351 1.00000000 0.08025977 

Reduced model m2

• Now that we’ve identified an initial set of variables to remove, we arrive at a reduced model (m2) in the form of:

\[\begin{align} logit[P(attrition = 1 (Yes))] = &\beta_0 + \beta_1age + \beta_2businesstravel + \\ &\beta_3dailyrate + \beta_5distancefromhome + \beta_6education + \\ &\beta_7educationfield + \beta_8environmentsatisfaction + \beta_{10}hourlyrate + \\ &\beta_{11}jobinvolvement + \beta_{14}jobsatisfaction + \\ &\beta_{15}maritalstatus + \beta_{17}monthlyrate + \beta_{18}numcompaniesworked + \\ &\beta_{19}overtime + \beta_{20}percentsalaryhike + \beta_{21}performancerating + \\ &\beta_{23}stockoptionlevel + \beta_{24}totalworkingyears + \beta_{25}trainingtimeslastyear + \\ &\beta_{28}yearsincurrentrole + \\ &\beta_{29}yearssincelastpromotion + \beta_{30}yearswithcurrmanager \end{align}\]

• Does the model (m2) fit?

• We see that model M2 fits by \(\frac{deviance_{res}}{df_{res}} \leq 1\). Check the marginal model plots to check model validity and to see if any of the continuous predictors are misspecified. If a predictor is misspecified, check the conditional density plots to see what kind of transformation may be needed.

Marginal Model Plots (scrollable)

• Check the marginal model plots (mmp) of the quantitative predictors to see if the model is specified correctly

Conditional Density Plots (scrollable)

• \(age\) appears to be misspecified.

• \(age\) appears to generally have a normal distribution and the same/similar variance for both values of \(attrition\) (i.e., yes and no). Let’s try adding a quadratic term to the model for \(age\).

Model m3

• Here we will include a quadratic term in the model for \(age\) and arrive at model (m3) in the form of:

\[\begin{align} logit[P(attrition = 1 (Yes))] = &\beta_0 + \beta_1age + \beta_{1a}age^2 + \beta_2businesstravel + \\ &\beta_3dailyrate + \beta_5distancefromhome + \beta_6education + \\ &\beta_7educationfield + \beta_8environmentsatisfaction + \beta_{10}hourlyrate + \\ &\beta_{11}jobinvolvement + \beta_{14}jobsatisfaction + \\ &\beta_{15}maritalstatus + \beta_{17}monthlyrate + \beta_{18}numcompaniesworked + \\ &\beta_{19}overtime + \beta_{20}percentsalaryhike + \beta_{21}performancerating + \\ &\beta_{23}stockoptionlevel + \beta_{24}totalworkingyears + \beta_{25}trainingtimeslastyear + \\ &\beta_{28}yearsincurrentrole + \\ &\beta_{29}yearssincelastpromotion + \beta_{30}yearswithcurrmanager \end{align}\]

• Does the model (m3) fit?

• We see that model m3 fits by \(\frac{deviance_{res}}{df_{res}} \leq 1\). Check the marginal model plots.

Marginal Model Plots (scrollable)

• Check the marginal model plots (mmp) of the quantitative predictors to see if the model is specified correctly

Observations on model m3

• Adding the \(age^2\) term corrects some misspecification in the model.

• Look at the standardized deviance residuals and inspect for outliers and bad leverage points

Leverage Plot

Observations

• There doesn’t appear to be any bad leverage points, although there are several points of high leverage.

• There does appear to be outliers in the data.

• Since this is a simulated dataset, we will assume that there is sufficient reason for removing the outliers.

Number of outliers

34

Outlier indices

6, 11, 17, 38, 47, 62, 68, 82, 154, 184, 204, 264, 309, 321, 352, 364, 390, 558, 561, 606, 609, 660, 668, 680, 685, 693, 720, 730, 755, 786, 817, 837, 876 and 910

Outlier data (scrollable)

Model m3 w/o outliers

• Does the model fit?

• We see that model m3 without outliers in the data still fits by \(\frac{deviance_{res}}{df_{res}} \leq 1\). Check the marginal model plots.

Marginal Model Plots (scrollable)

• Check the marginal model plots (mmp) of the quantitative predictors to see if the model is specified correctly

Observations on model m3

• After specifying the predictors correctly earlier, we saw that the linear fit of the model could, potentially, still improve. Therefore, we inspected the leverage and looked for outliers.

• We identified 34 outliers in the training set, and, for the purpose of this analysis, removed those outliers under the assumption that there was sufficient reason to do so. Recall, this is a simulated data set.

• After removing the outliers from the training set and re-running model m3 with the new data, we find that the overall linear fit of the model improved greatly.

• There still appears to be several predictors that are not significant, though. Let’s run step-wise variable selection to see if we can further reduce the number of predictors in the model.

Fwd/Bwd Stepwise Selection

Stepwise Model Path 
Analysis of Deviance Table

Initial Model:
attrition ~ age + I(age^2) + businesstravel + dailyrate + distancefromhome + 
    education + educationfield + environmentsatisfaction + hourlyrate + 
    jobinvolvement + jobsatisfaction + maritalstatus + monthlyrate + 
    numcompaniesworked + overtime + percentsalaryhike + performancerating + 
    stockoptionlevel + totalworkingyears + trainingtimeslastyear + 
    yearsincurrentrole + yearssincelastpromotion + yearswithcurrmanager

Final Model:
attrition ~ age + I(age^2) + businesstravel + dailyrate + distancefromhome + 
    educationfield + environmentsatisfaction + jobinvolvement + 
    jobsatisfaction + maritalstatus + numcompaniesworked + overtime + 
    totalworkingyears + trainingtimeslastyear + yearsincurrentrole + 
    yearssincelastpromotion


                    Step Df    Deviance Resid. Df Resid. Dev      AIC
1                                             965   366.5840 426.5840
2            - education  1 0.001671317       966   366.5857 424.5857
3          - monthlyrate  1 0.041329610       967   366.6270 422.6270
4           - hourlyrate  1 0.062737836       968   366.6898 420.6898
5    - performancerating  1 0.541018729       969   367.2308 419.2308
6    - percentsalaryhike  1 0.110578520       970   367.3414 417.3414
7     - stockoptionlevel  1 0.976428369       971   368.3178 416.3178
8 - yearswithcurrmanager  1 1.012913703       972   369.3307 415.3307

Observations

• Step-wise variable selection on model m3 removes seven variables.

• All seven variables removed from the model were not previously significant.

• Interestingly, \(dailyrate\) was not removed from the model even though it was not statistically significant before.

Notes

• Before applying the Lasso method, we consider the following:
  • \(gender\), \(relationshipsatisfaction\) and \(worklifebalance\) are removed from the model because of their independence from the response variable (\(attrition\)). See Logistic Regression \(\rightarrow\) Check X vs. Y Independence.
  • The 31 outliers identified in section Logistic Regression \(\rightarrow\) Leverage were removed from the training data set prior to reduce the effects of outliers on the model.
  • \(department\), \(joblevel\), \(jobrole\), \(monthlyincome\), and \(yearsatcompany\) were removed because of high VIF/GVIF values. See Logistic Regression \(\rightarrow\) Check VIFs.
  • The quadratic term \(age^2\) is added to the model because we saw earlier that \(age\) is misspecified in the model. See Logistic Regression \(\rightarrow\) Reduced Model(m2) and Reduced Model(m3) sections.
  • For the remaining predictor terms, we will apply the Lasso method for variable selection.
  • The starting model for SLR (lasso) is the same as model m3. See Logistic Regression \(\rightarrow\) Reduced Model(m3).

Model before applying SLR (lasso)

\[\begin{align} logit[P(attrition = 1 (Yes))] = &\beta_0 + \beta_1age + \beta_{1a}age^2 + \beta_2businesstravel + \\ &\beta_3dailyrate + \beta_5distancefromhome + \beta_6education + \\ &\beta_7educationfield + \beta_8environmentsatisfaction + \beta_{10}hourlyrate + \\ &\beta_{11}jobinvolvement + \beta_{14}jobsatisfaction + \\ &\beta_{15}maritalstatus + \beta_{17}monthlyrate + \beta_{18}numcompaniesworked + \\ &\beta_{19}overtime + \beta_{20}percentsalaryhike + \beta_{21}performancerating + \\ &\beta_{23}stockoptionlevel + \beta_{24}totalworkingyears + \beta_{25}trainingtimeslastyear + \\ &\beta_{28}yearsincurrentrole + \\ &\beta_{29}yearssincelastpromotion + \beta_{30}yearswithcurrmanager \end{align}\]

Sparse Logistic Regression (SLR) Model Fit

SLR Coefficients

Final Logistic Regr Coefficients by Step-wise Var Select

Final Logistic Regr Coef by Lasso Var Select

Random Forest Model

• We’ll use the whole training set for the Random Forest model. It should be less affected by colinearity.

• We’ll use the training set to identify important variables to keep and then re-run a more sparse model before measuring performance.

• We’ll also build two initial models based on: 1) training set w/ outliers and 2) training set w/o outliers

• For the plots to the right:

  • Green - class error for class 1 (i.e., \(attrition\) = 1 (yes))
  • Red - class error for class 0 (i.e., \(attrition\) = 0 (no))
  • Black - out of bag error
  • NOTE: we see lower error for class 0 because there are more “No” responses to learn from in the data

RF Model Plot (w/ outliers)

RF Model Plot (w/o outliers)

Variable Importance (w/ outliers) - Mean Decr in Accuracy

Variable Importance Plot (w/ outliers) - Mean Decr in Accuracy

Variable Importance (w/ outliers) - Mean Decr in Node Impurity

Variable Importance Plot (w/ outliers) - Mean Decr in Node Impurity

Variable Importance (w/o outliers) - Mean Decr in Accuracy

Variable Importance Plot (w/o outliers) - Mean Decr in Accuracy

Variable Importance (w/o outliers) - Mean Decr in Node Impurity

Variable Importance Plot (w/0 outliers) - Mean Decr in Node Impurity

Variable Importance (w/ outliers) - Mean Decr in Accuracy

Variable Importance Plot (w/ outliers) - Mean Decr in Accuracy

Variable Importance (w/ outliers) - Mean Decr in Node Impurity

Variable Importance Plot (w/ outliers) - Mean Decr in Node Impurity

Variable Importance (w/o outliers) - Mean Decr in Accuracy

Variable Importance Plot (w/o outliers) - Mean Decr in Accuracy

Variable Importance (w/o outliers) - Mean Decr in Node Impurity

Variable Importance Plot (w/0 outliers) - Mean Decr in Node Impurity

RF Reduced Model Plot (w/ outliers)

RF Reduced Model Plot (w/o outliers)

Performance - test set

Receiver Operating Curves (ROCs)